[Dataset Quality] Indicate if failure store isn't enabled for data stream #221644

rStelmach · 2025-05-27T15:23:48Z

Added a tooltip and a link with documentation for Failed docs column when dataset does not have failure store enabled.

failed_docs_tooltip.mov

For now it awaits for the documentation PR to be merged : elastic/docs-content#1368

…tore-isnt-enabled-for-data-stram

…-fix'

yngrdyn · 2025-05-29T10:59:32Z

/oblt-deploy

yngrdyn

Thanks for taking this one @rStelmach.
I have two main comments here:

I think we should change the approach here, from your changes I can infer that we are treating privileges (meaning user can not read failure store: read_failure_store) the same as a failure store not enabled for an specific dataStream. Take the following as an example (You can find everything configured in the PR deployment):

I've set up a custom component template enabling failure store for data streams following the pattern logs-foo-*

script

PUT _component_template/logs-foo@custom
{
  "template": {
    "data_stream_options": {
      "failure_store": {
        "enabled": true
      }
    }
  }
}

Then I ingested some documents in logs-foo-1 (Failure store enabled) and logs-bar-1 (Failure store disabled)

script

### Failure store enabled
POST logs-foo-1/_doc/1?op_type=create
{
  "foo": "bar"
}
POST logs-foo-1/_doc/2?op_type=create
{
  "foo": { "bar": 123}
}

#### Failure Store not enabled
POST logs-bar-1/_doc/1?op_type=create
{
  "foo": "bar"
}
POST logs-bar-1/_doc/2?op_type=create
{
  "foo": { "bar": 123}
}

User has proper privileges to read failure store over all datastreams following logs-* pattern

And this is what I see in dataset quality page

This is what I should be seeing

but I can only get to that state if the user doesn't have read privileges over failure store.

We should also consider what to do inside a specific dataStream in regards to failure Store, should we show something there (?) or instead we could use a banner at the top with a link to the specific documentation? (Something similar to what we do when user doesn't have privileges but now for enabling the failure store for the specific datastream)

@LucaWintergerst, @patpscal what are your thoughts on this one?

Note

I configured everything in the PR deployment, you can access with elastic, failure_store_some_enabled and failure_store_some_restricted with the same password generated by oblt-deploy command (🔒 deployment)

…s with/without failure store enabled

…tore-isnt-enabled-for-data-stram

…etails

…tore-isnt-enabled-for-data-stram

x-pack/platform/plugins/shared/dataset_quality/public/utils/generate_datasets.ts

x-pack/platform/plugins/shared/dataset_quality/public/utils/generate_datasets.test.ts

...platform/plugins/shared/dataset_quality/server/routes/data_streams/get_data_streams/index.ts

x-pack/test/functional/apps/dataset_quality/dataset_quality_table.ts

yngrdyn · 2025-06-11T10:08:15Z

We are missing some api tests (integration) asserting that we are getting the right information regarding failureStore being enabled/disabled

LucaWintergerst · 2025-06-11T14:17:04Z

@yngrdyn apologies for the super slow reply here - this is good, thank you work implementing it like this

patpascal · 2025-06-17T13:55:02Z

@patpscal is it ok to show the enablement as a warning? Should we treat it as an info instead?

I think we could treat that either way!

…lureStore`. remove unnecessary nullish coalescing operators

…-fix'

…tore-isnt-enabled-for-data-stram

yngrdyn · 2025-06-23T05:55:47Z

.../platform/plugins/shared/dataset_quality/public/components/dataset_quality/table/columns.tsx

+                          ? i18n.translate('xpack.datasetQuality.failureStore.enable', {
+                              defaultMessage: 'Set failure store',
+                            })
+                          : 'N/A'}


This should also be translated

yngrdyn · 2025-06-23T06:00:27Z

/oblt-deploy

yngrdyn · 2025-06-23T09:11:22Z

.../plugins/shared/dataset_quality/public/components/dataset_quality_details/overview/index.tsx

+                  css={{ textDecoration: 'underline' }}
+                >
+                  {i18n.translate('xpack.datasetQuality.enableFailureStore', {
+                    defaultMessage: 'Enable failure store',


Can we make this to look more like a link?

Adding an icon would be beneficial here @patpscal?

How about this ?

…tore-isnt-enabled-for-data-stram

elasticmachine · 2025-06-23T18:03:59Z

💚 Build Succeeded

Buildkite Build
Commit: f2bb67a

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`datasetQuality`	418.5KB	420.3KB	+1.7KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`datasetQuality`	23.8KB	24.1KB	+253.0B

History

💔 Build #311546 failed ed15c9a
💔 Build #311336 failed 1362929
💔 Build #310975 failed 66a42f1
💔 Build #310899 failed 77abd1c
💔 Build #310116 failed 257b60d
💔 Build #310013 failed 9f3fef1

yngrdyn

LGTM, thanks for taking this one
Good job 🚀

kibanamachine · 2025-06-24T07:49:50Z

Starting backport for target branches: 8.19

https://github.com/elastic/kibana/actions/runs/15844551689

…ream (elastic#221644) Added a tooltip and a link with documentation for Failed docs column when dataset does not have failure store enabled. https://github.com/user-attachments/assets/be65db9a-15c8-4087-b175-752b2fabab6e For now it awaits for the documentation PR to be merged : elastic/docs-content#1368 --------- Co-authored-by: kibanamachine <[email protected]> Co-authored-by: Yngrid Coello <[email protected]> (cherry picked from commit 36cbbb9)

kibanamachine · 2025-06-24T07:56:08Z

💚 All backports created successfully

Status	Branch	Result
✅	8.19

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

…data stream (#221644) (#225007) # Backport This will backport the following commits from `main` to `8.19`: - [[Dataset Quality] Indicate if failure store isn't enabled for data stream (#221644)](#221644)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport)  Co-authored-by: Robert Stelmach <[email protected]> Co-authored-by: Yngrid Coello <[email protected]>

…ream (elastic#221644) Added a tooltip and a link with documentation for Failed docs column when dataset does not have failure store enabled. https://github.com/user-attachments/assets/be65db9a-15c8-4087-b175-752b2fabab6e For now it awaits for the documentation PR to be merged : elastic/docs-content#1368 --------- Co-authored-by: kibanamachine <[email protected]> Co-authored-by: Yngrid Coello <[email protected]>

rStelmach added 2 commits May 27, 2025 17:08

initial version

631e69f

update documentation link

520379b

rStelmach changed the title ~~221006 dataset quality indicate if failure store isnt enabled for data stram~~ [Dataset Quality] Indicate if failure store isn't enabled for data stream May 28, 2025

rStelmach added 9.1 candidate v8.19.0 v9.1.0 release_note:feature Makes this part of the condensed release notes backport:version Backport to applied version labels Feature:Dataset Health and removed 9.1 candidate labels May 28, 2025

Merge branch 'main' into 221006-dataset-quality-indicate-if-failure-s…

38e0bfe

…tore-isnt-enabled-for-data-stram

rStelmach marked this pull request as ready for review May 28, 2025 15:53

rStelmach requested a review from a team as a code owner May 28, 2025 15:53

[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…

7839491

…-fix'

yngrdyn self-requested a review May 29, 2025 10:59

yngrdyn requested changes May 29, 2025

View reviewed changes

rStelmach and others added 11 commits May 30, 2025 15:43

add hasFailureStore flag to DataStreamStat model, distinguish dataset…

35cc59c

…s with/without failure store enabled

use passed hasFailureStore value instead of hardcoded false

eff4e3b

adjust tests

cfdedcd

Merge branch 'main' into 221006-dataset-quality-indicate-if-failure-s…

b10ef94

…tore-isnt-enabled-for-data-stram

adjust functional tests

243b2b3

add hasFailureStore to dataStreamStats

c63c5f4

remove unnecessary type

0597801

Merge branch 'main' into 221006-dataset-quality-indicate-if-failure-s…

1f654d9

…tore-isnt-enabled-for-data-stram

Check if failure store is enabled for datastream in dataset quality d…

677be06

…etails

Merge branch 'main' into 221006-dataset-quality-indicate-if-failure-s…

956d139

…tore-isnt-enabled-for-data-stram

adjust functional tests

43f94cc

rStelmach requested a review from yngrdyn June 9, 2025 10:58

Merge branch 'main' into 221006-dataset-quality-indicate-if-failure-s…

6fa40e7

…tore-isnt-enabled-for-data-stram

yngrdyn reviewed Jun 11, 2025

View reviewed changes

x-pack/platform/plugins/shared/dataset_quality/public/utils/generate_datasets.ts Outdated Show resolved Hide resolved

yngrdyn reviewed Jun 11, 2025

View reviewed changes

x-pack/platform/plugins/shared/dataset_quality/public/utils/generate_datasets.test.ts Show resolved Hide resolved

yngrdyn reviewed Jun 11, 2025

View reviewed changes

...platform/plugins/shared/dataset_quality/server/routes/data_streams/get_data_streams/index.ts Outdated Show resolved Hide resolved

yngrdyn reviewed Jun 11, 2025

View reviewed changes

x-pack/test/functional/apps/dataset_quality/dataset_quality_table.ts Show resolved Hide resolved

rStelmach and others added 6 commits June 18, 2025 13:11

add tests for hovering functionality. update tests to have `handleFai…

7f6db0e

…lureStore`. remove unnecessary nullish coalescing operators

[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…

d3358c6

…-fix'

Merge branch 'main' into 221006-dataset-quality-indicate-if-failure-s…

9f3fef1

…tore-isnt-enabled-for-data-stram

add api integration tests

257b60d

replace link to docs with datastream details locator

77abd1c

Merge branch 'main' into 221006-dataset-quality-indicate-if-failure-s…

66a42f1

…tore-isnt-enabled-for-data-stram

yngrdyn reviewed Jun 23, 2025

View reviewed changes

yngrdyn self-requested a review June 23, 2025 06:01

yngrdyn reviewed Jun 23, 2025

View reviewed changes

rStelmach and others added 4 commits June 23, 2025 11:46

resolve conflicts. add translation to 'N/A'. remove icon from warning

1362929

Merge branch 'main' into 221006-dataset-quality-indicate-if-failure-s…

ca558e8

…tore-isnt-enabled-for-data-stram

wait until tooltip dissapears in ftr

ed15c9a

move mouse back to the table after veryfing tooltip next

f2bb67a

yngrdyn approved these changes Jun 24, 2025

View reviewed changes

rStelmach merged commit 36cbbb9 into elastic:main Jun 24, 2025
10 checks passed

kibanamachine mentioned this pull request Jun 24, 2025

[8.19] [Dataset Quality] Indicate if failure store isn't enabled for data stream (#221644) #225007

Merged

[Dataset Quality] Indicate if failure store isn't enabled for data stream #221644

[Dataset Quality] Indicate if failure store isn't enabled for data stream #221644

Uh oh!

Conversation

rStelmach commented May 27, 2025 • edited by kibanamachine Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yngrdyn commented May 29, 2025

Uh oh!

yngrdyn left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

yngrdyn commented Jun 11, 2025

Uh oh!

LucaWintergerst commented Jun 11, 2025

Uh oh!

patpascal commented Jun 17, 2025

Uh oh!

yngrdyn Jun 23, 2025

Choose a reason for hiding this comment

Uh oh!

yngrdyn commented Jun 23, 2025

Uh oh!

yngrdyn Jun 23, 2025

Choose a reason for hiding this comment

Uh oh!

yngrdyn Jun 23, 2025

Choose a reason for hiding this comment

Uh oh!

rStelmach Jun 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Jun 23, 2025

💚 Build Succeeded

Metrics [docs]

Async chunks

Page load bundle

History

Uh oh!

yngrdyn left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kibanamachine commented Jun 24, 2025

Uh oh!

kibanamachine commented Jun 24, 2025

💚 All backports created successfully

Questions ?

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

rStelmach commented May 27, 2025 •

edited by kibanamachine

Loading

yngrdyn left a comment •

edited

Loading

rStelmach Jun 23, 2025 •

edited

Loading